Method for correcting motion vector errors caused by camera panning 

DESCRIPTION 



Background of Invention 

[Para 1] 1 . Field of the Invention 

[Para 2] The present invention relates to a method of correcting motion 
vectors, and more specifically, to a method of correcting motion vector errors 
in video frames caused by camera panning. 

[Para 3] 2. Description of the Prior Art 

[Para 4] Video frames are composed of a plurality of blocks. In some movies 
and films, camera-panning skills are used to take a panorama shot or to 
generate special filming effects. For instance, the MPEG-4 video coding 
standard is becoming a popular video format. In this video format, video 
frames are always partitioned into a set of blocks. Since there is information 
redundancy among video frames over time, one block in the current frame may 
be quite similar with some area of pixels in the previous frame. One motion 
vector is coded to present the position offset between them for each block in 
the current frame. 

[Para 5] Unfortunately, when transmitting video signals, errors may occur 
and cause a loss of some motion vectors. Blocks in the video frame that 
contain errors are referred to as error blocks. The presence of error blocks 
degrades the video quality, and leads to a less crisp picture. 
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Summary of Invention 



[Para 6] It Is therefore an objective of tlie claimed invention to provide a 
method of correcting motion vector errors experienced during camera panning 
In order to solve the above-mentioned problems. 

[Para 7] According to the claimed invention, a method of correcting errors in 
motion vectors due to camera panning includes identifying an error block that 
contains an error motion vector out of a plurality of blocks In a current video 
frame, each block In the current video frame containing a corresponding 
motion vector, calculating an average motion vector value of the motion 
vectors for all blocks in the current video frame that are not error blocks. 
Identifying a group of effective motion vectors by counting blocks in the 
current video frame having corresponding motion vector values within a first 
range of values bounded by the average motion vector value minus a first 
threshold value and the average motion vector value plus the first threshold 
value, calculating a panning motion vector value to be equal to an average 
value of the effective motion vectors, and correcting the error block by setting 
the motion vector of the error block to be equal to the panning motion vector 
value. 

[Para 8] It Is an advantage of the claimed invention that the motion vector of 
the error block can be corrected based on the motion vector values of the 
other blocks in the video frame. All of the blocks in the same video frame 
should have similar motion characteristics, and therefore the error block can 
be corrected with a high degree of confidence. 

[Para 9] These and other objectives of the present Invention will no doubt 
become obvious to those of ordinary skill in the art after reading the following 
detailed description of the preferred embodiment that is illustrated in the 
various figures and drawings. 
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Brief Description of Drawings 

[Para 1 0] Fig.1 is a diagram of a video frame containing error blocl<s. 

[Para 1 1] Fig. 2 is a flowcliart illustrating a method of correcting the error 
blocks according to a first embodiment of the present invention. 

[Para 1 2] Fig. 3 is a diagram of another video frame containing error blocks. 

[Para 1 3] Fig. 4 is a flowchart illustrating a method of correcting the error 
blocks according to a second embodiment of the present invention. 

Detailed Description 

[Para 14] The present invention aims to correct errors in motion vectors 
caused by camera panning. The present invention corrects the motion vector 
errors by replacing the motion vector of an error block with the panning vector 
value of the current video frame. 

[Para 1 5] The pixels and blocks in video frames shot using camera panning 
have the same or similar motion characteristics. The present invention relies 
on this characteristic to recover the motion vector of the error block. 

[Para 1 6] Please refer to Fig.1 and Fig. 2. Fig.1 is a diagram of a video frame 
1 0 containing error blocks 1 6. Fig. 2 is a flowchart illustrating a method of 
correcting the error blocks 16 according to a first embodiment of the present 
invention. The video frame contains a plurality of blocks, each of the blocks 
containing a respective motion vector. Steps contained in the flowchart will be 
explained below. 
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[Para 1 7] Step 1 00: Identify error blocks 1 6 that contain an error motion 
vector; 

[Para 1 8] Step 1 02: Calculate an average motion vector value AVG of all 
decoded motion vectors in the video frame 1 0. That is, for all of the blocks in 
the video frame 1 0 which are not error blocks 1 6, calculate the average of the 
motion vector values; 

[Para 1 9] Step 1 04: Identify a group of effective motion vectors 1 2. The 
effective motion vectors 1 2 are defined as those motion vectors falling in the 
range [AVG - THRESHOLDl , AVG + THRESHOLDl]. All other motion vectors can 
be labeled as non-effective motion vectors 14, meaning that they do not fall 
within this range; 

[Para 20] Step 106: Calculate a percentage of all the motion vectors in the 
video frame 1 0 that are effective motion vectors 1 2; 

[Para 21 ] Step 1 08: Determine if the percentage of effective motion vectors 
1 2 is above a reference value; if so, go to step 1 1 0; if not, go to step 1 1 6; 

[Para 22] Step 1 1 0: Set a panning motion vector value PANNING to be equal 
to the average of all effective motion vector values 1 2; 

[Para 23] Step 112: Determine if the panning motion vector value PANNING is 
suitable for correcting the error block 1 6. This can be performed by calculating 
an average motion vector value or a median motion vector value of four or 
eight blocks surrounding the error block 16 to be corrected. If the average or 
median motion vector value for the surrounding blocks is within the range 
[PANNING -THRESH0LD2, PANNING + THRESH0LD2], the panning motion 
vector value PANNING is said to be suitable for correcting the motion vector of 
the error block 1 6, and the flow proceeds to step 1 1 4; Otherwise, go to step 
116; 

[Para 24] Step 1 1 4: Correct the motion vector of the error block 1 6 to have 
the panning motion vector value PANNING; and 

[Para 25] Step 1 1 6: Use another method for correcting the motion vector of 
the error block 1 6. 
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[Para 26] As shown in Fig.l , the motion vectors of the three error blocl<s 1 6 
can all be corrected to be the panning motion vector value PANNING. This of 
course assumes that the percentage of effective motion vectors 1 2 is above a 
reference value and that the four or eight blocks surrounding each of the error 
blocks 1 6 have motion vector values within a second threshold value 
THRESH0LD2 of the panning motion vector value PANNING. The percentage of 
effective motion vectors 1 2 is preferably set equal to 70-90%. Additionally, the 
first and second thresholds THRESHOLDl and THRESH0LD2 can have values of 
between 0.5 and 3. Of course, these values are only provided for illustrative 
purposes, and by no means limit the scope of the present invention. 

[Para 27] In the first embodiment of the present invention described above, 
the video frame 1 0 is taken as a whole, and is not divided into sections for 
correcting each of the error blocks 16. Therefore, the error blocks 16 that are 
corrected will be corrected to have their motion vector values set equal to the 
panning motion vector value PANNING. 

[Para 28] Please refer to Fig. 3 and Fig. 4. Fig. 3 is a diagram of a video frame 
40 containing error blocks 56, 66, and 76. Fig.4 is a flowchart illustrating a 
method of correcting the error blocks 56, 66, and 76 according to a second 
embodiment of the present invention. Unlike the first embodiment, the second 
embodiment updates panning motion vector values with each iteration of the 
method. Steps contained in the flowchart will be explained below. 

[Para 29] Step 1 50: Identify a first error block 56 that contains an error 
motion vector. Also, identify a first section 50 of blocks that is located before 
the first error block 56. The first section 50 of blocks are blocks located in 
rows above the first error block 56 and the blocks located to the left of the 
first error block 56 in the same row as the first error block 56; 

[Para 30] Step 1 52: Calculate an average motion vector value AVGl of all 
decoded motion vectors in the first section 50; 
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[Para 31] Step 1 54: Identify a first group of effective motion vectors 52. TPie 
effective motion vectors 52 are defined as tliose motion vectors in tPie first 
section 50 falling in the range [AVGl - THRESHOLDl , AVGl + THRESHOLDl]. 
All other motion vectors in the first section 50 can be labeled as non-effective 
motion vectors 54, meaning that they do not fall within this range; 

[Para 32] Step 1 56: Calculate a percentage of all the motion vectors in the 
first section 50 of the video frame 40 that are effective motion vectors 52; 

[Para 33] Step 1 58: Determine if the percentage of effective motion vectors 
52 is above a reference value; if so, go to step 1 60; if not, go to step 1 78; 

[Para 34] Step 1 60: Set a first panning motion vector value PANNING(1 ) to be 
equal to the average of all effective motion vector values 52 in the first section 
50; 

[Para 35] Step 162: Determine if the first panning motion vector value 
PANNING(l) is suitable for correcting the first error block 56. This can be 
performed by calculating an average motion vector value or a median motion 
vector value of four or eight blocks surrounding the first error block 56. If the 
average or median motion vector value for the surrounding blocks is within the 
range [PANNING(l) - THRESH0LD2, PANNING(l) + THRESH0LD2], the first 
panning motion vector value PANNING(l) is said to be suitable for correcting 
the motion vector of the first error block 56, and the flow proceeds to step 
1 64; Otherwise, go to step 1 78; 

[Para 36] Step 1 64: Correct the motion vector of the first error block 56 to 
have the first panning motion vector value PANNING(l); 

[Para 37] Step 1 66: Determine If there Is another error block in the video 
frame 40; if so, go to step 1 68; if not, end the process; 

[Para 38] Step 168: Identify an Nth error block 66, 76 that contains an error 
motion vector. The variable N represents an integer greater than or equal to 2 
since the first error block has already been corrected at this point. Also, 
identify an Nth section 60, 70 of blocks, each of which is located before the 
Nth error block 66, 76. In the example shown in Fig. 3, N can equal 2 or 3 since 
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the video frame 40 in Fig. 3 only contains a second section 60 and a tPiird 
section 70; 

[Para 39] Step 1 70: Identify an Nth group of effective motion vectors 62, 72. 
The effective motion vectors 62, 72 are defined as those motion vectors in the 
Nth section 60, 70 falling in the range [PANNING(N-1 ) - THRESH0LD2, 
PANNING(N-l) + THRESH0LD2]. All other motion vectors in the Nth section 60, 
70 can be labeled as non-effective motion vectors 64, 74, meaning that they 
do not fall within this range; 

[Para 40] Step 1 72: Set an Nth panning motion vector value PANNING(N) to be 
equal to the average of all effective motion vector values 52, 62, 72 in the first 
N sections 50, 60, 70; 

[Para 41] Step 174: Determine if the Nth panning motion vector value 
PANNING(N) is suitable for correcting the Nth error block 66, 76. This can be 
performed by calculating an average motion vector value or a median motion 
vector value of four or eight blocks surrounding the Nth error block 66, 76. If 
the average or median motion vector value for the surrounding blocks is within 
the range [PANNING(N) - THRESH0LD2, PANNINC(N) + THRESH0LD2], the Nth 
panning motion vector value PANNING(N) is said to be suitable for correcting 
the motion vector of the Nth error block 66, 76, and the flow proceeds to step 
1 76; Otherwise, go to step 1 78; 

[Para 42] Step 1 76: Correct the motion vector of the Nth error block 66, 76 to 
have the Nth panning motion vector value PANNING(N), go back to step 1 66; 

[Para 43] Step 1 78: Use another method for correcting the motion vector of 
the current error block. 

[Para 44] As shown in Fig. 3, the three sections 50, 60, and 70 of the video 
frame 40 precede the three error blocks 56, 66, 76. Since there are only three 
error blocks 56, 66, 76 shown in the video frame 40, a plurality of blocks 80 
located after the third error block 76 are not used for correcting the motion 
vector values of the error blocks 56, 66, 76. By dividing the video frame 40 
into a plurality of sections, the various error blocks contained in the video 
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frame 40 can each be corrected with unique motion vector values, thereby 
providing a more realistic and accurate correction for the error blocks. 
Conveniently, the same example threshold values and example percentage 
used in the first embodiment of the present invention may also be used with 
the second embodiment, although these are not intended as limiting. 

[Para 45] In steps 112,1 62, and 1 74 above, the motion vector values in four 
or eight neighboring macro blocks (MBs) of the error MB are used to decide 
whether the panning motion vector (MV) value is suitable for correcting the 
error MB. 

[Para 46] As described in video standards, the MV value is set for each block 
only in the Y channel and no MV is set for the blocks in the U and V channels. 
Therefore, the motion vector info for the block in the U and V channels is 
calculated from that in the Y channel. 

[Para 47] Since one MB in the Y channel contains four blocks, which are the 
smallest available unit for MV coding, the four MV values are coded for one 
MB. When an error occurs in one MB and causes the loss of MV info, the four 
MVs for that MB will all be lost as a result of the error propagation in the video 
bit stream. Therefore, in the example in which four neighboring MBs are used, 
1 6 total MVs from the four neighboring MBs are used for the decision of 
panning MV suitability. Similarly, 32 total MVs can be used if eight neighboring 
MBs are used instead of four neighboring MBs. 

[Para 48] In summary, the present invention corrects the motion vector of an 
error block based on the motion vector values of the other blocks in the video 
frame. Since all of the blocks in the same video frame have similar motion 
characteristics, the error block can be corrected with a high degree of 
confidence. The present invention method is both simple to implement and 
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improves the overall video quality considerably when errors are present in 
motion vectors caused by camera panning in video frames. 

[Para 49] Those skilled in the art will readily observe that numerous 
modifications and alterations of the device and method may be made while 
retaining the teachings of the invention. Accordingly, the above disclosure 
should be construed as limited only by the metes and bounds of the appended 
claims. 
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